A lazy bagging approach to classification

In this paper, we propose lazy bagging (LB), which builds bootstrap replicate bags based on the characteristics of test instances. Upon receiving a test instance xk, LB trims bootstrap bags by taking into consideration xk’s nearest neighbors in the training data. Our hypothesis is that an unlabeled instance’s nearest neighbors provide valuable information to enhance local learning and generate a classifier with refined decision boundaries emphasizing the test instance’s surrounding region. In particular, by taking full advantage of xk’s nearest neighbors, classifiers are able to reduce classification bias and variance when classifying xk. As a result, LB, which is built on these classifiers, can significantly reduce classification error, compared with the traditional bagging (TB) approach. To investigate LB’s performance, we first use carefully designed synthetic datasets to gain insight into why LB works and under which conditions it can outperform TB. We then test LB against four rival algorithms on a large suite of 35 real-world benchmark datasets using a variety of statistical tests. Empirical results confirm that LB can statistically significantly outperform alternative methods in terms of reducing classification error.